Generally, a “machine” is a system or device that performs or assists in the performance of at least one task. Completing a task often requires the machine to collect, process, and/or output information, possibly in the form of work. For example, a vehicle may have a machine (e.g., a computer) that is designed to continuously collect data from a particular part of the vehicle and responsively notify the driver in case of detected adverse vehicle or driving conditions. However, such a machine is not “intelligent” in that it is designed to operate according to a strict set of rules and instructions predefined in the machine. In other words, a non-intelligent machine is designed to operate deterministically; should, for example, the machine receive an input that is outside the set of inputs it is designed to recognize, the machine is likely to, if at all, generate an output or perform work in a manner that is not helpfully responsive to the novel input.
In an attempt to greatly expand the range of tasks performable by machines, designers have endeavored to build machines that are “intelligent,” i.e., more human- or brain-like in the way they operate and perform tasks, regardless of whether the results of the tasks are tangible. This objective of designing and building intelligent machines necessarily requires that such machines be able to “learn” and, in some cases, is predicated on a believed structure and operation of the human brain. “Machine learning” refers to the ability of a machine to autonomously infer and continuously self-improve through experience, analytical observation, and/or other means.
Machine learning has generally been thought of and attempted to be implemented in one of two contexts: artificial intelligence and neural networks. Artificial intelligence, at least conventionally, is not concerned with the workings of the human brain and is instead dependent on algorithmic solutions (e.g., a computer program) to replicate particular human acts and/or behaviors. A machine designed according to conventional artificial intelligence principles may be, for example, one that through programming is able to consider all possible moves and effects thereof in a game of chess between itself and a human.
Neural networks attempt to mimic certain human brain behavior by using individual processing elements that are interconnected by adjustable connections. The individual processing elements in a neural network are intended to represent neurons in the human brain, and the connections in the neural network are intended to represent synapses between the neurons. Each individual processing element has a transfer function, typically non-linear, that generates an output value based on the input values applied to the individual processing element. Initially, a neural network is “trained” with a known set of inputs and associated outputs. Such training builds and associates strengths with connections between the individual processing elements of the neural network. Once trained, a neural network presented with a novel input set may generate an appropriate output based on the connection characteristics of the neural network.
Some systems have multiple processing elements whose execution needs to be coordinated and scheduled to ensure data dependency requirements are satisfied. Conventional solutions to this scheduling problem utilize a central coordinator that schedules each processing element to ensure that data dependency requirements are met, or a Bulk Synchronous Parallel execution model that requires global synchronization.